A data playback system (which will be simply referred to herein as a “player”) for reading compressed and encoded video and audio signals from their storage medium such as a CD-ROM, a DVD-ROM, or a hard disk, decoding (or expanding) the signals and then outputting decoded data is known. When such a player sends the expanded data to a monitor or a loudspeaker, which is connected to the system, the playback of video and audio is realized.
A lot of standards have been defined as to compression coding. For example, an MPEG (Motion Picture Experts Group) standard is known as a standard for compressing and encoding both a video signal and an audio signal alike. According to an MPEG standard, to guarantee synchronous playback of video represented by a video signal and audio represented by an audio signal, time information, representing the time to present it, is added to each of the audio and video signals. Then, by making reference to these pieces of time information with respect to the system clock reference of the system equipment during expansion, the audio and video signals can be played back synchronously with each other.
Hereinafter, a conventional synchronizing method will be described. For example, according to the technique disclosed in Japanese Patent Application Laid-Open Publication No. 6-233269, first, the player senses the difference between a point in time when one of audio and video has been expanded and a point in time when the other has been expanded. Then, based on the magnitude of the difference sensed, the player changes the timing to transfer compressed data, the timing to activate an expander, and the timing to present the video, thereby synchronizing the audio and video together and outputting them to a monitor, a loudspeaker, and other external units.
However, this player outputs the audio and video to the external units after having synchronized them with each other internally. That is why if one of those external units processed the audio or video to produce some delay, then one of the audio and video could not be played back synchronously with the other anymore.
For example, suppose the external unit is a video display unit such as a TV set. In that case, if the video display unit performed any conversion processing such as an interlace/progressive conversion or an NTSC/PAL conversion or any image processing (such as magnifying or reducing the size of the video) at the user's request, then it would take a while to get such processing done. As a result, the playback of audio would precede that of video. For instance, if image processing to magnify the output image was requested, then some computations should be made to produce interpolated data from the original output image data. In that case, to generate smooth moving pictures, filter operations should be performed on the basis of a plurality of temporally continuous output frames or fields. Also, supposing the processing of combining two or more video streams output from the same circuit that can expand a plurality of incoming streams independently of each other, the video output of a particular stream might have to wait in order to synchronize all of those video streams together. Furthermore, in converting a video signal into an optical signal once and then transmitting the optical signal as in a theater system, it might take some time to get the video signal converted.
It might take several to several tens of milliseconds to get one, two or more of such computation processing done. If the playback of audio preceded that of video by a certain amount of time (approximately 30 milliseconds) or more, then the lag between the mouth movement of a person in the video and the output audio would be so noticeable as to make the viewer uncomfortable. The conventional player pays no attention at all to such a presentation time lag between audio and video caused by the processing performed by an external unit. That is why if the external unit carried out processing that would take an even longer time, then the video presentation time would be way behind the audio presentation time.
It is not impossible to play back the audio and the processed video synchronously with each other by providing the external unit with a buffer for causing a lag in the audio playback. In that case, however, the video data and audio data that have been output to the external unit have no time information to specify their presentation times, and the external unit should determine the output timings by itself, which is not an appropriate measure to take. Also, this technique of providing a buffer is not applicable to an existent external unit accepting no buffers, and therefore, cannot be called an ultimate solution.
In order to overcome the problems described above, an object of the present invention is to play back audio and video synchronously with each other just as originally intended even if the lag of the video playback varies according to the type of the external unit connected or the type of the processing adopted.